: Computers and Symbols versus Nets and Neurons 5 Non-binary Signal Communication N Time Slots N 1 '1's or Pulses Probability of a '1' or Pulse Approx. N 1 / N Pulse Stream 2: Tlus and Vectors -simple Learning Rules

1 These notes are currently under review for publication by UCL Press Limited in the UK. Duplication of this draft is permitted by individuals for personal use only. Any other form of duplication or reproduction requires prior written permission of the author. This statement must be easily visible on the rst page of any reproduced copies. I would be happy to receive any comments you might have on this draft; send them to me via electronic mail at [email protected]. I am particularly interested in hearing about things that you found diicult to learn or that weren't adequately explained, but I am also interested in hearing about inaccuracies, typos, or any other constructive criticism you might have. 1 1 Neural net: A preliminary deenition To set the scene it is useful to give a deenition of what we mean by`Neural Net'. However, it is the object of the course to make clear the terms used in this deenition and to expand considerably on its content. A Neural Network is an interconnected assembly of simple processing elements, units or nodes, whose functionality is loosely based on the animal neuron. The processing ability of the network is stored in the inter-unit connection strengths, or weights, obtained by a process of adaptation to, or learning from, a set of training patterns. In order to see how very diierent this is from the processing done by conventional computers it is worth examining the underlying principles that lie at the heart of all such machines. 2 The von Neumann machine and the symbolic paradigm The operation of all conventional computers may be modelled in the following way memory central processing unit instructions and data data von-Neumann machine The computer repeatedly performs the following cycle of events 1. fetch an instruction from memory. 2. fetch any data required by the instruction from memory. 3. execute the instruction (process the data). 4. store results in memory. 5. go back to step 1). 2 What problems can this method solve easily? It is possible to formalise many problems in terms of an algorithm, that is as a well deened procedure or recipe which will guarantee the answer. For example, the solution to a set of equations or the way to search for an item in a database. This algorithm may then be broken down into a set of simpler statements which can, in turn, be reduced eventually, to the …